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.... of a parallel computer for a given algorithm What is the best algorithm and architecture for solving a 
problem as the problem size and number of processors change A number of performance evaluation 
metrics have been developed to study the scalability of parallel algorithms and architectures [3, 4, 10, 11, 
12]. Kumar and Gupta [7] provide a compre hensive survey of different methods of scalability analysis. The 
isoefficiency function is one such metric. It relates the size of the problem being solved to the number of 
processors required to maintain the efficiency at a fixed value. An important feature .... 
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(Network General Circulation Model) Their analysis shows that contention does affect .... 

J. L. Gustafson, "The Consequences of Fixed Time Performance Measurement, " in Proceedings of the Twenty- 
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....problem to 13 increase at the rate necessary to maintain a fixed efficiency, then the parallel system should be 
considered unscalable from a practical point of view. 2. 3 Relationship between Isoefficiency and Other 
Metrics A number of scalability metrics have been proposed by various researchers [22, 34, 36, 55, 61, 59, 
60, 75, 80, 98, 108, 131, 130, 132, 137, 142, 146, 149]. We present a detailed survey of these metrics in [84] 
After reviewing these various measures of scalability, one may ask whether there exists one measure that is 
better than all others [66] The answer to this question is no, as different measures are suitable for different 
situations. One .... 
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....is half as fast. From these results, it is inferred that it is better to have a parallel computer with fewer faster 
processors than one with many slower processors. We discuss this issue further in Section 7 and show that this 
inference is valid only for a certain class of parallel systems. In [49, 21], Sun and Gustafson argue that 
traditional speedup is an unfair measure, as it favors slow processors and poorly coded programs. They 
derive some fair performance measures for parallel programs to correct this. The authors define sizeup as the 
ratio of the size of the problem solved on the .... 

....serial operation is same but M 1 executes the parallelizable operations faster than M 2 . They show that M 1 
will attain poorer speedups (even if the communication overheads are ignored) but according to sizeup, M 1 will 
be considered better. Based on the concept of fixed time sizeup, Gustafson [21] has developed the 
SLALOM benchmark for a distortion free comparison of computers of widely varying speeds. Sun and 
Rover [51] define a scalability metric called the isospeed measure, which is the factor by which the problem size 
has to be increased so that the average unit speed of computation .... 

John L. Gustafson. The consequences of fixed time performance measurement. In Proceedings of the 25th 
Hawaii International Conference on System Sciences: Volume III, pages 113-124, 1992. 



Parallel Best-First Branch-and-Bound in Discrete.. - Correa, Ferreira (1995) (3 citations) (Correct) 

.... This phenomenon is also one of the explanations for speedup anomalies (see section 4) Gustafson 
cites the parallel B B as a counterexample to the affirmation that the best sequential algorithm defines the 
least work necessary , which he considers as a myth of performance measurement [21]. The definition of e 
(p) considers this myth as a fact to measure the computational parallel efficiency of parallel B B algorithms. In the 
following, we shall define parallel efficiency measures which take into account the two factors involved in the 
efficiency of parallel synchronous B B .... 

J. Gustafson. The consequences of fixed time performance measurement. In 25th Hawaii Int. Conf. on Syst. Sc., 
volume III. IEEE, Jan. 1992. 



The Performance and Scalability of Parallel Systems - Davies (1994) (Correct) 

.... to performance measures such as efficiency [36, 38, 40, 68, 101] For others the performance issue is 
the reduction in the runtime (as measured by the elapsed time or wall clock ) for a given problem [8, 31 , 
39, 40, 44, 91] or the running of the largest feasible problem on the available system [54, 55, 116]; These 
two differing views lead to the models of fixed size speedup and scaled speedup respectively. As can be 

seen from just these few examples, there are differing views of this term performance. In several of the proposed 
models the qualities being measured are implicit, in that there is not .... 

....the sequential portion of any algorithm s computation is difficult. Amdahl [8] saw this serial portion as part of 
the inevitable housekeeping ; others [39, 70, 116, 126] have seen it as an inescapable constant overhead or 
some function of the number of processors in the system. Gustafson viewed [54] it as a function of both the 
number of processors and the ensemble size the granularity of the executed problem. The observation [8, 
51, 55] that most algorithms have some inherently sequential portion, that does not yield to parallelisation, 
appears to be perfectly reasonable. This is .... 
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John L. Gustafson. The consequences of fixed time performance measurement. In Scriver [110]. 



Isoefficiency Function: A Scalability Metric for Parallel.. - Ananth Grama (1993) (10 citations) (Correct) 

.... of a parallel computer for a given algorithm What is the best algorithm and architecture for solving a 
problem as the problem size and number of processors change A number of performance evaluation 
metrics have been developed to study the scalability of parallel algorithms and architectures [3, 4, 10, 11, 

1 2]. Kumar and Gupta [7] provide a comprehensive survey of different methods of scalability analysis. The 
isoefficiency function is one such metric. It relates the size of the problem being solved to the number of 
processors required to maintain the efficiency at a fixed value. An important feature .... 

... used be d dimensional (i:e: p = 2 d ) and let n = 2 r . An interesting property of the mapping shown in 
Figure 5 is that the vector elements residing on different processors are combined during the first d 
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iterations, while the pairs of elements combined during the X[0] X[1] X[2] X[3] X[4] X[5] X[6] X[7] X[8] X[9] 
X[10] X[11] X[12] X[13] X[14] X[15] P 3 P 2 P 1 P 0 Y[0] Y[1] Y[2] Y[3] Y[4] Y[5] Y[6] Y[7] Y[8] Y[9] Y[10] Y[11] 
Y[12] Y[13] Y[14] Y[15] m = 0 m = 1 m = 2m = 3 Figure 5: A 16 point FFT on four processors. P i denotes 
processor number i and m refers to the .... 
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Hawaii International Conference on System Sciences: Volume III, pages 113-124, 1992. 
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....bound for the speedup, and show that there is no theoretical upper limit on heterogeneous speedup. 1 
Introduction Program speedup is a widely used measure of the performance of an algorithm on a multiprocessor 
or multicomputer. Although there are differing notions of the definition of speedup [7], speedup 
measurements are still almost universally quoted as proof of system efficiency. A common definition of 
speedup is that if machine M 1 can solve problem P in time T 1 , and machine M 2 solves the same problem in 
time T 2 , then the speedup S P of machine M 2 over machine M 1 on P is the .... 

....M 2 achieves linear speedup over machine M 1 , whereas if S P m, the speedup This research was supported 
in part by NSF Grants ASC 9106465 and ASC 9301788. is sublinear. Superlinear speedup, where S P m, is 
especially interesting and somewhat controversial. Quinn [15] and Gustafson [7] discuss arguments for and 
against the possibility of superlinear speedup. A heterogeneous network is one in which multiple, possibly 
dissimilar, machines cooperate in solving a problem. The intuitive advantage of a heterogeneous network is the 
ability to match segments of code with machines in a .... 

Gustafson, J.L. The consequences of fixed time performance measurement. Proc. 25th Hawaii International 
Conference on System Sciences. Jan. 1 992, pp. 1 1 3-1 24. 



Parametric Micro-level Performance Models for Parallel Computing - Y oungtae Kim (1994) (1 citation) (Correct ) 

....The papers [19, 3] discuss shortcomings of earlier theoretical research, and propose new models called BSP 
and LogP for parallel computation. An important aspect of both models is the incorporation of communication 
parameters which were ignored in earlier theoretical research. The studies [2, 7, 8, 18] address several 
pragmatic issues and provide insights into important attributes of parallel performance. A good 
introduction to performance and scalability of parallel systems is provided in recent books [10, 12] This paper is 
about parametric micro level (PM) performance models for parallel .... 

....which incorporates precise details of interprocessor communication, memory operations, miscellaneous 
overheads due to auxiliary instructions, and effects of communication and computation schedules. Execution 
times can be predicted by fitting timing curves to experimental data, as discussed in [8]. The basic 
approach is to determine an algebraic expression for the fitting formula by analysis of algorithm and then 
determine the coefficients by experiments. This approach is closely aligned with our goals; it can accurately 
predict execution times. A fitting formula expresses execution time .... 

[Article contains additional citation context not shown here] 

Gustafson, J. L. The Consequences of Fixed Time Performance Measurement. Proc. Twenty-fifth Hawaii Internat. 
Conf. System Sciences Vol.3. 1992, pp. 113124. 



Scheduling From the Perspective of the A pplication - Berman, Wolski (1996) (49 citations) (Correct) 

....trade off the time required to skim distributed data to make'it local with the time required for data access from 
remote locations. In general, performance criteria vary with the user and the application. Most users employ 
common criteria such as execution time, speedup (fixed size or fixed time [12]) cost of execution cycles, 
etc. although performance goals vary considerably over metacomputing applications. Moreover, distinct users 
will attempt to optimize their usage of same metacomputing resources for different performance criteria at the 
same time. For individual applications, the .... 
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GUSTAFSON, J. The consequences of fixed time performance measurement. Proceedings of the 25th Hawaii 
International Conference on System Sciences (Jan 1992), 113-124. 



Knowledge Of Characteristics In Multiproqrammed Multiprocessor.. - Parsons (1997) (Correct) 

....resulting in better speedup. In order to provide a fairer comparison, Gustafson proposed sizeup as the ratio of 
the size of problem that can be solved on a parallel computer to that on a uniprocessor within a fixed amount of 
time. This sizeup metric led to the development of the SLALOM benchmark [Gus92] A more recent 
scalability metric involves determining the rate at which a problem size must increase in order to 
maintain a certain level of efficiency as the number of processors is increased. The 2. BACKGROUND 38 
function that characterizes the problem size increase is called the .... 

John L. Gustafson. The consequences of fixed time performance measurement. In Proceedings of the 25th 
Hawaii Conference on System Sciences, volume III, pages 113- 124, 1992. 



A Parallel Workload Model and Its Implications for Processor.. - Allen Downey (1 996) (41 citations) (Correct) 

....function of the problem size. Larger problems generally have more available parallelism, ffl Users often submit 
jobs with problem size and cluster size chosen such that the turnaround time is less than some constant time 
limit. Thus, the cluster size and the problem size tend to increase together [6]. ffl In the aggregate, we 
expect the shape of the distribution of chosen cluster sizes to reflect the shape of the distribution of average 
parallelism. Figure 4 shows the distribution of cluster sizes for the workloads from SDSC and CTC. In both cases, 
almost all jobs have cluster sizes that are .... 

John L. Gustafson. The consequences of fixed time performance measurement. In Proceedings of the Twenty- 
Fifth Hawaii International Conference on system Sciences, Vol. Ill, January 1992. 



Performance Properties of Large Scale Parallel Systems - A. Gupta, et al. (1993) (1 3 citations) (Correct) 

....to thousands of processors are commercially available today and offer substantially higher raw computing 
power than the fastest sequential supercomputers. Availability of such systems has fueled interest in 
investigating the performance of parallel computers containing a large number of processors [23, 8, 7, 27, 
37, 30, 6, 33, 18, 32, 17, 38, 34, 4, 5, 29]. The performance of a parallel algorithm cannot be studied in isolation 
from the parallel architecture it is implemented on. For the purpose of performance evaluation we define a parallel 
system as a combination of a parallel algorithm and a parallel architecture on which it is implemented 

John L. Gustafson. The consequences of fixed time performance measurement. In Proceedings of the 25th 
Hawaii International Conference on System Sciences: Volume III, pages 113-124, 1992. 



NetPIPE: A Network Protocol Independent Performance Evaluator - Quinn Snell Armin (1 996) (12 citations) 
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....the transfer block size k from a single byte until transmission time exceeds 1 second. Hence, NetPIPE is a 
variable time benchmark and will scale to all network speeds. Unlike fixed size benchmark tests, NetPIPE will 
not become outdated and inaccurate as technology advances (see Gustafson [6]) To increase the 
universality of NetPIPE, information is measured in bits rather than bytes. The definition of byte varies more 
than one might think. For each block size c, three measurements are taken: c Gamma p bytes, c bytes, and c p 
bytes, where p is a perturbation factor with a default .... 

Gustafson, J. "The Consequences of Fixed Time Performance Measurement' , Proceedings of the 25th Annual 
Hawaii International Conference on Systems Sciences, IEEE Computer Society Press, Vol 3, pages 113-124. 
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